Modeling Motion of Body Parts for Action Recognition

Human action recognition is a challenging problem that has received considerable attention from the computer vision community in recent years. Its applications are diverse, spanning from its use in activity understanding for intelligent surveillance systems to improving human-computer interactions. Ideally, the desired representation for actions should generalize over variations in viewpoint, human appearance, and spatio-temporal changes. Human action representation can be divided into two categories: global representations and local representations [3]. The global representations can encode much of the information but they are more sensitive to the environment. Local representations are less sensitive to the environment but they depend on the accuracy of interest point detectors. In this paper, we propose a generative representation of the motion of the human body parts to learn and classify human actions. The proposed representation combines the advantages of both local and global representations, encoding the relevant motion information as well as being robust to local appearance changes. Our work is motivated by the pictorial structures model [1] and the framework of sparse representations for recognition [4]. Human part movements are represented efficiently through quantization in the polar space. The key discrimination within each action is efficiently encoded by sparse representation to perform classification. Figure 1 depicts the overview of our approach. We propose a novel use of 2D histograms of body part locations in polar geometry as a representation of human action. Collectively, all the 2D histograms of each body part’s location generated over the entire video forms a description of the relative motion of body parts that constitutes a specific action. The motion descriptor of each body part is a 2D histogram of size R×O, where R and O are the numbers of radial and orientation bins, respectively. The 2D histogram is treated as a R×O motion descriptor image. Thus, every human action is represented by P motion descriptor images, each describing the motion of the P body parts. Let us define the set of K human action classes to be recognized. A basic problem in action recognition is to determine the class that a new test sample belongs to. Let us consider a set of nk training videos for the kth action class where each video results in P motion descriptor images. As a result, for a particular human body part p, we will have a set of nk training samples from the kth class as columns of a matrix A k = [a p k,1, ...,a p k,nk ]∈Rm×nk , where each motion descriptor image of part p is identified as the vector ap ∈ Rm(m = R×O). Let us consider the testing video y resulting in P motion descriptor images which are identified as the set of P vectors {yp ∈Rm | p= 1, ...,P}. For a particular human body part p, we want to represent test sample yp as a sparse linear combination of training samples. Given sufficient training samples of the kth action class, A k = [a p k,1, ...,a p k,nk ]∈Rm×nk , any new test sample yp ∈Rm from the same class approximately lies in the linear span of the training samples associated with action class k; i.e.,